Benchmarking comprises techniques for measuring performance of a certain aspect of circuitry and/or software. These techniques generally involve executing a series of benchmarking operations and measuring performance characteristics associated with those operations in order to directly or indirectly infer performance characteristics of the circuitry and/or software being measured. The benchmarking operations may be performed in a series of distinct benchmarking tests, with individual benchmarking results obtained for each benchmarking test. For a variety of reasons, such as operations executing in the background, benchmarking results for different benchmarking tests may differ. Thus, to obtain benchmarking results that are deemed to be sufficiently “accurate” or representative of performance over varying background conditions, a large number of benchmarking tests are performed and the benchmarking results obtained are combined.
One aspect of benchmarking is determining the number of benchmarking tests to perform in order to obtain a sufficiently “accurate” set of benchmarking results. It is important to choose a large enough number of benchmarking tests in order to reach a high enough confidence level that the results obtained are sufficiently representative of performance. However, performing too many benchmarking tests is wasteful.